Improved probability estimation with neural network models

Wei Wei, Etienne Barnard and Mark Fanty Center for Spoken Language Understanding Oregon Graduate Institute of Science and Technology 20000 N.W.Walker Road, Portland, OR 97291-1000 ABSTRACT Neural network classi ers can provide outputs that estimate Bayesian posterior probabilities under the assumptions that an in nite amount of training data are available, the network is su ciently complex and the training can reach the global minimum. In practice, however, the number of training tokens is limited and may not accurately re ect the prior class probabilities and true likelihood distributions. Additionally, computational constraints place a limit on the complexity of the network. Consequently, practical networks often fall far short of being ideal estimators. We address this problem and propose a new method of improved probability estimation by combining neural network models with empirical probability estimation methods. We use a histogram-based estimation method to remap the network outputs to match the data and thereby improve the accuracy of the probability estimates. Our current experiments on the OGI Census Year corpus resulted in a 20.6% reduction in recognition errors at the utterance level.